Pulsed Laser-Based Firearm Training System, and Method for Facilitating Firearm Training Using Detection of Laser Pulse Impingement of Projected Target Images

ABSTRACT

The invention provides a method for automatic calibration and subsequent correlation of the position of a pulsed laser on a projected image in a system having a projector for projecting images onto a surface and a camera for sensing laser pulses on the surface. The automatic calibration method sets the camera exposure, allowing the system to operate in normal room-lighting conditions, and correlates camera pixel positions to projected image pixel positions by use of projected calibration images formed by sets of horizontal and vertical lines, with automatic calibration completing in less than 5 seconds. After calibration, the system determines two-dimensional camera pixel centroids of laser beam pulses on the projection surface to sub-pixel accuracy, and the calibration data is used to correlate the camera pixel centroids to the exact positions of the laser pulses on the projected image.

PRIORITY CLAIMED

Benefit is claimed to provisional application No. 61/076,048 filed onJun. 26, 2008.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT ANDDEDICATORY CLAUSE

This invention was made with Government support under a contract awardedby the U.S. Army (Agreement # W31P4Q-05-A-0031 and Subcontract#4600005773). The invention described herein may be manufactured, usedand licensed by or for the U.S. Government for governmental purposeswithout payment of any royalties thereon.

FIELD OF THE INVENTION

This invention relates to firearm training and more specifically tofirearm arm training using a pulsed laser against projected targetimages to determine hit points.

BACKGROUND OF THE INVENTION

Firearms are utilized for a variety of purposes, such as hunting,sporting competition, law enforcement, and military operations. Theinherent danger associated with firearms necessitates training andpractice in order to minimize the risk of injury. However, specialfacilities are required to facilitate practice of handling and shootingthe firearm.

These special facilities tend to provide a sufficiently sized area forfirearm training, where the area required for training may become quitelarge, especially for sniper-type or other firearm training withextended range targets. The facilities further confine projectilespropelled from the firearm within a prescribed space, thereby preventingharm to the surrounding environment. Accordingly, firearm trainees arerequired to travel to the special facilities in order to participate ina training session, while the training sessions themselves may becomequite expensive since each session requires new ammunition forpracticing handling and shooting of the firearm.

In addition, firearm training is generally conducted by severalorganizations (e.g., military, law enforcement, firing ranges or clubs,etc.). Each of these organizations may have specific techniques ormanners in which to conduct firearm training and/or qualify trainees.Accordingly, these organizations tend to utilize different types oftargets, or may utilize a common target, but with different scoringcriteria. Furthermore, different targets may be employed by users forfirearm training or qualification to simulate particular conditions orprovide a specific type of training (e.g., grouping shots, hunting, claypigeons, etc.).

Prior systems have been provided, using video cameras to provide targettracking. For example, U.S. Pat. No. 5,366,229 (Suzuki) discloses ashooting game machine including a projector for projecting a video imageonto a screen, wherein the video image includes a target. In this game,it is the goal of a player to fire a laser gun to emit a light beam atthe target displayed in the video image on the screen.

In the Suzuki '229 patent, a video camera photographs the screen andprovides a picture signal to coordinate computing means for determiningthe X and Y coordinates of the beam point on the screen (so as todetermine whether the light beam struck the target). Such systems, forinstance, utilize measurement of the luminance and/or chromance of avideo image to determine where, within the target image, a targetappears.

International Publication No. WO 92/08093 (Kunnecke et al.) discloses asmall arms target practice monitoring system including a weapon, atarget, a light-beam projector mounted on the weapon and sighted topoint at the target, and a processor. An evaluating unit is connected toa camera to determine the coordinates of the spot of light (projected bythe light beam projector) on the target. A processor is connected to theevaluating unit and receives the coordinate information. The processorfurther displays the spot on the projected target image displayed on adisplay screen.

Such systems may calculate a position for the image, for instance byreference to a region defined by matching chromance and/or luminance topattern values, which might match patterns for flesh tones and/or otherapplicable patterns. Another type of video tracking system relies on aspecified window to isolate regions of interest in order to determine atarget. Analog comparison techniques may be used to perform tracking.

Although the above conventional target tracking systems can track alaser beam impingement on a projected video image, calibration of suchsystems are tedious and inefficient, frequently requiring manualcalibration of same to ensure that the projected image is properlycorrelated with the sensing device and computer system.

The accuracy of any such system can never be better than the calibrationor alignment between the projected images and the laser beam sensingsystem. For example, U.S. Pat. No. 6,616,452, calls for the computersystem to perform a mechanical calibration and a system calibrationbefore training/simulation may begin. The mechanical calibrationgenerally facilitates alignment of the sensing device with the projectedimage (generally projected onto a screen) and computer system, while thesystem calibration enables determination of parameters for systemoperation.

In particular, in the system described in U.S. Pat. No. 6,616,452, thecomputer system displays a calibration graphical user screen, and theuser must then adjust the displayed coordinates. The computer systemcompensates for the device viewing angle offset, and requests the userto indicate, preferably via a mouse or other input device, the cornersof the projected image within the captured images within a window of thecalibration screen. The coordinates for a corner designated by a userare displayed on the screen, where the user may selectively adjust thecoordinates.

This process is repeated for each corner of the projected image todefine for the computer system the projected image within the imagecaptured by the camera. The horizontal and vertical lines are adjustedin accordance with the entered information to indicate the systemperspective of the projected image. This calibration is then repeateduntil horizontal lines are substantially coincident with thecorresponding projected image horizontal edges, and the projected imagehorizontal center line and vertical line are substantially coincidentwith the vertical center line of the captured image, thereby indicatingalignment of the projected image with image captured (recorded) by thecamera of the system.

Such conventional calibration means and methods are very time consuming,of limited accuracy, and are dependent upon user interaction throughoutthe process. Therefore, there is a need for a system for firearmtraining capable of precise target tracking with concomitant ease ofcalibration to correlate camera pixels to target pixels of the projectedtarget image in various light and environmental conditions.

SUMMARY OF THE INVENTION

In one embodiment of the present invention, a method for automaticcalibration of a laser-based firearm training system having a projectorfor projecting images onto a surface and a camera for reading imagesprojected onto the surface is provided. The method includes projecting afirst image comprised of an all white screen upon the target surface,reading the first image and adjusting an exposure setting of the camerabased on the first image so as to ensure pixel values below saturation.

The method further includes projecting a second image comprising a blackcolor upon the surface, reading the second image and calculating abackground light level value for each pixel in the second image. Themethod further includes projecting a third image comprising a pattern ofhorizontal lines upon the surface, reading the third image andcalculating a horizontal calibration value for each pixel in the thirdimage, wherein the resulting horizontal calibration value for each pixelis reduced by the corresponding background light level value.

The method further includes projecting a fourth image comprising apattern of vertical lines upon the surface, reading the fourth image andcalculating a vertical calibration value for each pixel in the fourthimage, wherein the resulting vertical calibration value for each pixelis reduced by the corresponding background light level value. The methodfurther includes calculating a centroid line for each vertical andhorizontal line and calculating a second order line least squares fitfor each centroid line, thereby generating three coefficients thatdescribe each vertical and horizontal line and generating a data map formapping a pixel from an image read from the surface to a pixel in animage projected into the surface, wherein the data map is based on thethree coefficients that describe each vertical and horizontal line.

In another embodiment of the present invention, a method for automaticdetection of a laser pulse in a laser-based firearm training systemhaving a projector for projecting images onto a surface and a camera forreading images projected onto the surface is provided. The methodincludes reading a first projected image into a first data structure andsearching for a laser pulse in the image by searching for pixels in thefirst data structure with a pixel value exceeding a predefinedthreshold.

The method further includes selecting a set of pixels around a pixelhaving a pixel value exceeding a predefined threshold and calculating acentroid of the laser pulse in the first data structure by performing atwo dimensional centroid calculation upon the set of pixels. The methodfurther includes calculating a pixel location of the laser pulse in thefirst image based on the centroid.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter, which is regarded as the invention, is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features and also theadvantages of the invention will be apparent from the following detaileddescription taken in conjunction with the accompanying drawings.

FIG. 1 is a box diagram of the pulsed laser-based firearm trainingsystem of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

A pulsed laser-based firearm training system, and method of utilizingsame is provided, wherein a laser beam pulse is “fired” at a projectedtarget image, and a video camera, in conjunction with hardware andsoftware means, determines the location of the laser beam strike on theprojected target image by correlation of the projected target imagepixels to the video camera pixels. The hardware and software means, inconjunction with a video projector and video camera, is capable ofautomatically correlating the video camera pixels (i.e., the image“captured” by the system) to the projected target image pixels (i.e.,the projected image).

As illustrated in FIG. 1, the present invention provides a pulsedlaser-based firearm training system 1 comprised of a projection means 3,a video camera means 5, and a hardware-software processing means 7. Inaddition, a laser emitting device 9 is provided. The system 1 isoperable to detect the aim points of weapons, having the laser emittingdevice mounted thereon or integrated therewith, that are fired at aprojected target image 11.

The performance of the system of the present invention is relativelyindependent of the accuracy of setup of the video camera means 5 withrespect to the projected target image, and can be used with nodegradation in high and uneven ambient lighting conditions, as long asthe projected image is not completely overcome by the ambient light.Importantly, the system 1 of the present invention, via a softwareprogram (i.e., the automatic calibration module 13, as illustrated) isexecuted on the processing means 7, automatically calibrates itself inless than 3 seconds, i.e., correlates camera pixels (i.e., pixelsgarnered by the video camera means 5) to target pixels of the projectedtarget image 11 (i.e., pixels comprising the image that is projected asimage 11) without the need for human interaction.

In operation, the system 1 is first calibrated via the automaticcalibration module 13 and method. Then, a simulated or actual weapon(usually any rifle or pistol), equipped with the laser emitting device 9(aligned with the sights or offset a predetermined amount from thesights thereof), is fired at the projected target image 11, triggeringthe laser emitting device 9 to pulse a dot laser-pulse 20 on the image11.

The system 1 detects the positions of the laser pulse 20 impinging onthe projected target image 11. Then, the system 1 correlates thepositions of the detected laser pulse 20 on the projected image to thecorresponding pixel of the image captured by the camera, based on therelation of the projected image to the captured image calculated in theautomatic calibration process mentioned above. This process is performedin real time (less than 7 milliseconds when a 210 Frame Per Second videocamera is used as the video camera means 5).

The laser emitting device 9 may be any commercially available visible ornear infrared dot laser. For example, a 3 milliwatt commercial laseremitter, which is readily available, performs well in the presentapplication. However, the laser can be any visible laser or a near IRlaser that is visible to the camera, with a minimum power of about 1.5milliwatts. In the laser emitting device 9 of the present invention, alaser pulse circuit applies power to the laser when the weapon is“fired,” and is referred to as a “fire event”. The laser pulse lengthmust be at least one video frame plus 2 milliseconds long (for a 210frames per second video camera, the pulse length should be at least 6.8milliseconds long).

The means of detecting the fire events may vary, including, but notlimited to, a trigger contact, a microphone sensing circuit, and amagnetic proximity circuit. The fire event is usually detected audiblywhen a real weapon is used in “dry fire” mode, i.e., when the impact ofthe hammer is heard, a fire event is generated. Various means can beused to detect fire events in both simulated and actual weapons, such asa switch in the trigger mechanism, magnetic reed switch or Hall Effectsensor. The laser emitting device 9 is preferably configured such thatthe laser circuit causes a small LED to blink when it fires the laser,so as to provide a visible indication when an infrared (IR) laser isused.

The projection means 3 may be any conventional video projector, as longas the strength of the projected image does not exceed that of the laserpulse, or, if the projected target image is too bright, a “long pass”optical filter attached to the camera lens fixes this problem; forexample, a 550 nm long pass filter allows the auto calibration to work,but causes the projected image seen by the video camera to be subduedabout 50% without substantially affecting the brightness of the laser.

In the preferred embodiment, the 550 nm long pass filter was foundsuitable with all projectors tested. The processing means 7 (i.e., acomputer system) may be comprised of any conventional computer system,such as a conventional IBM-compatible laptop or other type of personalcomputer (e.g., notebook, desk top, mini-tower, Apple Macintosh, palmpilot, etc.), preferably equipped with a keyboard and a mouse.

The computer system (processing means 7) may utilize any of the majorplatforms (e.g., Linux, Macintosh, Unix, OS2, etc.), but preferablyincludes a Windows environment (e.g., Windows XP or Vista). Further, theprocessing means 7 may include other conventional components (e.g.processor, disk storage or hard drive, etc.) having sufficientprocessing and storage capabilities to effectively execute the systemsoftware.

The computer system is in communication with the set-up parameters ofthe video camera means 5 via, for example, a USB (Universal Serial Bus)interface to set frame exposure time and other camera configurationparameters. The video camera means 5 may be mounted on a tripod, table,component rack, etc., and positioned at a suitable location from thesurface upon which the target image is to be projected. However, anytype of mounting or other structure may be utilized to support the videocamera means 5. The video camera means 5 is typically implemented by acamera employing a CMOS (complementary-symmetrymetal-oxide-semiconductor) imager. For example, any conventionalcommercially available video camera may be used.

However, preferably, a video camera capable of operating at 210 framesper second or greater is used. The use of a higher speed camera candetect shots fired faster. The high speed video camera also necessitatesa higher speed interface to the FPGA (Field Programmable Gate Array),such as the CameraLink standard, which also reduces the required numberof interface connections. The preferred implementation is to use acommercial off-the shelf (COTS) video camera and a CameraLink interfaceto an external (from the camera), standalone FPGA processing board.

The processing means 7 (and the FPGA therein), in conjunction with thevideo camera means 5, and the calibration constants, detects the camerapixel location of the laser beam impact on the target image (e.g., bycapturing an image of the target surface and detecting the location ofthe laser beam pulse impact from the captured image), and includes asignal processor (FPGA) and associated circuitry to provide impactlocation information in the form of X and Y camera coordinates to theprocessing means 7, or provide other data to the processing means 7 toenable determination of those camera coordinates.

As called for in the first embodiment herein, the processing meanscomprises an automatic calibration module (i.e., a computer softwareprogram) operable to correlate camera pixels to target pixels of theprojected target image. In particular, the automatic calibration modulecorrelates the target image pixels to the pixels of the video camerameans 5, to enable the processing means 7, in conjunction with the videocamera means 5, to correlate the location of a laser beam impact on thetarget image as an X and Y camera coordinate.

The resulting camera coordinates are transmitted to the processing means7 for translation to coordinates within the computer system's scaledtarget space, to determine the laser beam impact location intarget-pixel X and Y coordinates, as described below.

In practice, the processing means first generates a projected targetimage onto a “target” surface (such as a screen, but other surfaces,such as walls, may be used) that allows the video projector to be set upin zoom, focus, and keyhole adjustment. The preferred projected targetimage, in terms of ease of camera setup, has short line features aroundthe periphery of the projected image, and inside that target image isdisplayed whatever the video camera sees. The operator can command theprocessing means (e.g., a PC, which sends commands to the camera throughthe FPGA board) to set the video image brighter or darker, as needed forbest viewing by the operator. This allows the video camera to be set upin alignment, zoom, and focus.

Then, the calibration procedure is begun with the projector and camerain focus, and the projected image framed in the camera's view acceptablywell to the eye. When the video image adjustments are complete, theoperator commands the processing means (i.e., the computer system 7) toexecute the automatic calibration module 13 to correlate the videocamera pixels with the projected target image pixels.

The steps in the automatic calibration process are as follows. First, awhite screen image is projected upon the surface, so as to project araster image. In particular, a white screen is displayed with all pixelsset to color white and at maximum intensity. Then, the image is read bythe camera 5. Subsequently, the camera exposure is automaticallyadjusted by commands to the camera from the PC until no pixel read bycamera 5 is at saturation.

Subsequently, a black image is projected by projector 3 upon thesurface, for a plurality of successive frames. Then, multiple successiveframes of the image are read by the processing means and a backgroundlight level value is saved for each camera 5 pixel using an average fromthe multiple frames that were read. The background light level valuesfor each pixel are saved in a data structure such as a matrix or twodimensional array such that the two-coordinate address for each arrayelement is identical to the two-coordinate address for a pixel in theimage read by camera 5. Each array element includes a background lightlevel value calculated for the corresponding pixel read by camera 5.

Then, projector 3 projects a pattern of white horizontal calibrationlines upon the surface, with the center line missing. Then, the image isread by the processing means and the previously saved background imageis subtracted to yield a differential image that contains only thecalibration lines. The differential image of the horizontal calibrationraster values for each pixel may be stored in a two-dimensional arraysuch that the two-coordinate address for each array element is identicalto the two-coordinate address for a pixel in the image read by camera 5.Each array element includes a horizontal calibration raster valuecalculated for the corresponding pixel read by camera 5.

Then, projector 3 projects a pattern of equally spaced white verticalcalibration lines upon the surface, with center line missing. Then, theimage is read by the processing means and the previously savedbackground image is subtracted to yield a differential image thatcontains only the calibration lines.

The differential image of the vertical calibration raster values foreach pixel may be stored in a two-dimensional array such that thetwo-coordinate address for each array element is identical to thetwo-coordinate address for a pixel in the image read by camera 5. Eacharray element includes a vertical calibration raster value calculatedfor the corresponding pixel read by camera 5.

The computer 7 scans the differential calibration line rasters and firstfinds the identity of each calibration line from the missing line nearthe center of the calibration rasters. The calibration lines have bothwidth and noise as received from the camera, and the camera pixels thatform the calibration lines are composed of analog values that containrandom noise components. The first step in using the calibration linesis to determine the exact center line of each calibration line. A linecentroid calculation is used to determine the exact center of eachcalibration line and to remove noise, thus providing a new calibrationline to be used when performing a least squares fit for each calibrationline.

A line centroid calculation is performed for each camera pixel along thelength of each calibration line by selecting a reference point a couplepixels away from the line and then, moving toward and across the line toits far side, multiplying the pixel value of each pixel encountered bythe pixel distance from the reference point, summing all the products,dividing that sum by the total of all pixels encountered, and adding thequotient to the pixel position of the reference pixel, thus arriving atthe exact fractional pixel value of the center of the line width at onepixel position along the length of the calibration line.

The centroid process is repeated until centroid fractional pixel valueshave been determined for all pixels along the length of the calibrationline. The locus of centroids now forms a new calibration line that runsdown the exact center of the original calibration line that was composedof many camera pixels. A second order least-squares fit of the newcalibration line yields an accurate correlation of camera pixels totarget pixels along that calibration line, and the original calibrationline composed of many analog pixel values has been reduced to threecoefficients for a second order line.

Finally, a data map is generated for mapping a pixel from an image readfrom the surface (camera pixel) to a pixel in an image projected intothe surface (target pixel), wherein the data map is based on the threecoefficients that describe each vertical and horizontal line.

The automatic calibration method of the present invention yieldssuperior accuracy because it involves thousands of pixels percalibration line to determine the exact positions of the displayedcalibration lines. The line centroid calculations use pixels adjacent toboth sides of the line to obtain the maximum use of available lineposition data. One set of three second-order line coefficients aredetermined from a least-squares fit for each calibration line that wasprojected on the screen. By nature, a least-squares fit for the secondorder equations of the calibration lines virtually ignores single pixelerrors along the calibration lines.

The automatic calibration procedure spans less than 3 seconds of time,and yields sub-pixel accurate correlations between camera pixels andprojected target pixels.

For example, a projector projecting 1024 by 768 pixels is used toproject the target image, with some rows of projected pixels off-screen(i.e., off the target surface), i.e., some of the target image projectedboth above the top and below the bottom of the screen. A video camera isutilized, having a recording capability of 640 by 480 pixels. Cameraexposure is controlled by setting the exposure time per frame. Note thatmore calibration lines could be implemented, but testing proved that theaccuracy of the correlation is already at the sub-pixel level, andtherefore not needed.

Upon completion of the automatic calibration process described above,the shot detection and correlation module (target mode) may be entered.In the “target mode”, the computer 7 no longer needs to see the cameraraster. The computer 7 commands the FPGA to enter the “target mode” tostart looking for laser pulses in each camera frame. In target mode, thecamera exposure is set much lower than in the calibration mode describedabove, such that the video camera sees little to none of the projectedimages on the screen. Therefore, the laser pulse energy level mustexceed the maximum energy level of the projected target image, or alow-pass filter can be placed over the camera lens to block a portion ofthe visible projected target image while allowing the laser pulse topass through to the camera's imager with little attenuation.

The FPGA analyzes each camera raster image, looking for any signal levelthat exceeds a certain threshold (which happens when a laser pulse ispresent in a video frame). The FPGA then determines the exact centroidof the laser pulse in camera pixels, using all pixels in a large “WindowOf Interest” (WOI) of the camera imager.

The centroid calculations are performed as follows. For example, supposethe chart below contains a set of 8×8 pixel values of a laser pulse on aprojected target image.

47 28 29 31 34 30 25 52 31 87 42 42 46 46 98 33 37 45 214 254 254 254 4437 29 45 151 254 254 122 45 34 40 47 165 254 254 143 54 36 27 58 167 123137 230 52 31 40 55 34 39 42 34 69 39 32 27 35 31 36 34 28 36

First, in order to determine the X centroid value, take the sum ofcolumn one, and multiply it times its position in the matrix, which isone:

1*(47+31+37+29+40+27+40+32)=283

Next, take the sum of column two, and multiply it times its position inthe matrix, which is two:

2*(28+87+45+45+47+58+55+27)=784

Next, take the sum of column three, and multiply it times it position inthe matrix, which is three:

3*(29+42+214+151+165+167+34+35)=2511

Continue this process until all eight columns have been accounted for,and sum up the resulting values:

283+784+2511+4112+5285+5358+2905+2384=23622

Now, sum up the entire matrix, which is 5203, and the X centroid valueis calculated as

23622/5203=4.5401, a sub-pixel accurate result.

This X centroid value is the position within the matrix itself In orderto determine the overall X centroid value, the relative position of thematrix, as shown in the chart above, within the video frame should beaccounted for.

The Y centroid value is computed in the same way, except summing acrossrows instead of columns, with the result equal to 4.3686.

The preferred embodiment WOI is 16×16 camera pixels, such that theentire laser pulse is contained inside the WOI. The FPGA processes theWOI and calculates the camera pixel coordinates of the laser pulse tosub-camera-pixel accuracy, and forwards the resulting laser pulseposition in camera coordinates to the computer 7. The computer 7converts the camera pixel coordinates into projected target pixelcoordinates using the second order line coefficients that werecalculated in the automatic calibration procedure. The targetcoordinates are used to determine where the shot struck the targetimages.

In a preferable example, a Xilinx® Virtex-4 FPGA (Field ProgrammableGate Array) receives the camera image through a standard CameraLinkinterface. The FPGA analyzes each camera frame to determine if a laserpulse has occurred by searching for a camera pixel whose value exceedsan adjustable threshold, hereinafter referred to as a trigger-pixel.When found, a Window Of Interest (WOI) is framed around that point inorder to capture information from the entire laser pulse.

The preferred WOI is 16×16 pixels in size for a 640×480 pixel camera,with the top-left corner designated at coordinates (1,1), thetrigger-pixel designated at coordinates (8,8), and the bottom-rightcorner designated at coordinates (16,16). A circular buffer of videolines is maintained in FPGA memory so that once the trigger-pixel hasbeen located, the first 6×16 pixels of the WOI are retrieved frommemory.

The remaining ten lines of the WOI are then filled from the live videostream, relative to the location of the trigger-pixel. This methodaccommodates various shapes of focused laser pulses, while maintainingan ambient-light margin around the WOI edge.

Once the WOI has been captured, a two-dimensional centroid calculationis performed on the result in order to determine the exact camera-pixelposition of the laser pulse to a fraction of a pixel in both the X and Ydirections. The two-dimensional centroid calculation is achieved byaccumulating weighted sums based on column and row positions, dividingby the sum of the WOI, and adding offsets based on the WOI locationwithin the camera frame.

In particular, the two-dimensional centroid calculation proceeds asfollows: Each pixel value corresponds to the intensity of light seen bythat camera pixel. First, the ambient light-level is accounted for bytaking an average of the pixels around the edge of the WOI (the firstand last columns, and the first and last rows), and then subtractingthis result from each pixel in the WOI. Experimentally, this has provento increase accuracy over processing the WOI unaltered. It is importantthat the laser pulse be sized so that it does not hit in the frame edgeof the WOI, in order to not artificially raise the ambient light-level.

Subsequently, the X-direction centroid is determined by multiplying eachpixel in the WOI by its Y-direction pixel number (1 to 16), summing all256 products, and dividing the sum by the sum of all pixel values in theWOI. The Y-direction centroid is determined by multiplying each pixel inthe WOI by its X-direction pixel number (1 to 16), summing all 256products, and dividing the sum by the sum of all pixel values in theWOI. This process provides camera coordinates of the laser pulse tosub-pixel accuracy.

The resulting camera pixel position is forwarded to the processing meansfor implementation in the shot correlation.

In practice, the FPGA provides a laser pulse position in camera pixels.This is converted to target pixels, using the second order curveconstants that were determined during the automatic calibration moduleexecution described above. In order to find the target pixel Xcoordinate using the preferred method, the processing means calculatesthe X positional coordinates of the vertical lines using the previouslydetermined calculation constants along the row of the Y position givenby the FPGA. With these X positions, the processing means thendetermines the two closest positions to the given X coordinate from theFPGA.

These two positions of the vertical lines in the camera frame correlateto the same positions used in the target frame. The processing meansuses these line coordinates to determine the target pixel X coordinatethrough linear interpolation—the target pixel is positioned relative tothe two vertical lines in the target frame the same as the given camerapixel is positioned relative to the two vertical lines in the cameraframe. In order to find the target pixel Y coordinate, the same stepsare taken using the horizontal lines.

Lastly, the resulting X and Y pixel coordinates are scaled by the targetresolution over the camera resolution to determine the actual targetpixel position. Note that further processing could have been usedinstead of linear interpolation to yield a slightly more accurate targetpixel position of the laser pulse, but it was found by testing that thelinear interpolation readily yields single pixel accuracy that isadequate for our immediate needs.

Although specific embodiments of the present invention have beendisclosed herein, those having ordinary skill in the art will understandthat changes can be made to the specific embodiments without departingfrom the spirit and scope of the invention. The scope of the inventionis not to be restricted, therefore, to the specific embodiments.Furthermore, it is intended that the appended claims cover any and allsuch applications, modifications, and embodiments within the scope ofthe present invention.

1. A method for automatic calibration of a laser-based firearm trainingsystem having a projector for projecting images onto a surface and acamera for reading images projected onto the surface, comprising:projecting a first image comprised of horizontal and vertical lines uponthe surface; reading the first image; adjusting an exposure setting ofthe camera based on the first image so as to ensure pixel values belowsaturation; projecting a second image comprising a black color upon thesurface; reading the second image; calculating a background light levelvalue for each pixel in the second image; projecting a third imagecomprising a pattern of horizontal lines upon the surface; reading thethird image; calculating a horizontal calibration value for each pixelin the third image, wherein the resulting horizontal calibration valuefor each pixel is reduced by the corresponding background light levelvalue; projecting a fourth image comprising a pattern of vertical linesupon the surface; reading the fourth image; calculating a verticalcalibration value for each pixel in the fourth image, wherein theresulting vertical calibration value for each pixel is reduced by thecorresponding background light level value; calculating a centroid linefor each vertical and horizontal line and calculating a second orderline least squares fit for each centroid line, thereby generating threecoefficients that describe each vertical and horizontal line; andgenerating a data map for mapping a pixel from an image read from thesurface to a pixel in an image projected into the surface, wherein thedata map is based on the three coefficients that describe each verticaland horizontal line.
 2. The method of claim 1, wherein the first imageis set to the color white.
 3. The method of claim 1, wherein the firstimage is set to the color red.
 4. The method of claim 1, wherein thefirst step of calculating further comprises: storing the backgroundlight level value for each pixel in a two dimensional array.
 5. Themethod of claim 4, wherein the second step of calculating furthercomprises: storing the horizontal calibration value for each pixel in atwo dimensional array.
 6. The method of claim 5, wherein the third stepof calculating further comprises: storing the vertical calibration valuefor each pixel in a two dimensional array.
 7. A method for automaticdetection of a laser pulse in a laser-based firearm training systemhaving a projector for projecting images onto a surface and a camera forreading images projected onto the surface, comprising: reading a firstprojected image into a first data structure; searching for a laser pulsein the image by searching for pixels in the first data structure with apixel value exceeding a predefined threshold; selecting a set of pixelsaround a pixel having a pixel value exceeding a predefined threshold;calculating a centroid of the laser pulse in the first data structure byperforming a two dimensional centroid calculation upon the set ofpixels; and calculating a pixel location of the laser pulse in the firstimage based on the centroid.
 8. The method of claim 7, wherein the setof pixels comprises a set of 16×16 pixels centered at a pixel having apixel value exceeding a predefined threshold.
 9. The method of claim 8,wherein the centroid is a two-dimensional centroid.
 10. The method ofclaim 9, wherein the second step of calculating further comprises:calculating an X-direction centroid.
 11. The method of claim 10, whereinthe step of calculating an X-direction centroid further comprises:multiplying each pixel in the set of pixels by an X-direction coordinateof the pixel.
 12. The method of claim 11, wherein the step ofcalculating an X-direction centroid further comprises: summing theproduct of multiplying each pixel in the set of pixels by an X-directioncoordinate of the pixel so as to produce a sum.
 13. The method of claim11, wherein the step of calculating an X-direction centroid furthercomprises: dividing the sum by the sum of all pixel values in the set ofpixels.
 14. The method of claim 9, wherein the second step ofcalculating further comprises: calculating a Y-direction centroid. 15.The method of claim 14, wherein the step of calculating a Y-directioncentroid further comprises: multiplying each pixel in the set of pixelsby a Y-direction coordinate of the pixel.
 16. The method of claim 15,wherein the step of calculating a Y-direction centroid furthercomprises: summing the product of multiplying each pixel in the set ofpixels by a Y-direction coordinate of the pixel so as to produce a sum;and dividing the sum by the sum of all pixel values in the set ofpixels.
 17. A firearm laser training system operable to detect thelocation of a projected laser beam pulse upon a projected target imageis provided, said system comprising: a projection means operable toproject a target image upon a surface; a video camera means operable toscan said target image to produce scanned images of said target image,including impact locations of said laser beam pulse on said targetimage; and a processing means operable to receive from said video camerameans information associated with said impact locations detected by saidvideo camera means, said processing means comprising: an automaticcalibration module operable to correlate camera pixels to target pixelsof the projected target image; a detection module operable to determinea camera pixel position of the laser beam pulse; and a shot correlationmodule operable to correlate the camera pixel position of the laser beampulse to the projected target image, so as to determine the accuracy ofthe shot.
 18. The firearm laser training system of claim 17, wherein theautomatic calibration module comprises: a means operable to project afirst image comprised of horizontal and vertical lines upon the surface;a means operable to read the first image; a means operable to adjust anexposure setting of the camera based on the first image, so as to ensurepixel values below saturation; a means operable to project a secondimage comprising a black color upon the surface; a means operable toread the second image; a means operable to calculate a background lightlevel value for each pixel in the second image; a means operable toproject a third image comprising a pattern of horizontal lines upon thesurface; a means operable to read the third image; a means operable tocalculate a horizontal calibration value for each pixel in the thirdimage, wherein the resulting horizontal calibration value for each pixelis reduced by the corresponding background light level value; a meansoperable to project a fourth image comprising a pattern of verticallines upon the surface; a means operable to read the fourth image; ameans operable to calculate a vertical calibration value for each pixelin the fourth image, wherein the resulting vertical calibration valuefor each pixel is reduced by the corresponding background light levelvalue; a means operable to calculate a centroid line for each verticaland horizontal line and calculating a second order line least squaresfit for each centroid line, thereby generating three coefficients thatdescribe each vertical and horizontal line; and a means operable togenerate a data map for mapping a pixel from an image read from thesurface to a pixel in an image projected into the surface, wherein thedata map is based on the three coefficients that describe each verticaland horizontal line.
 19. The firearm laser training system of claim 18,wherein the first image is set to the color white.
 20. The firearm lasertraining system of claim 18, wherein the first image is set to the colorred.
 21. The firearm laser training system of claim 18, wherein themeans operable to calculate the background light level for each pixel inthe second image further comprises: a means operable to store thebackground light level value for each pixel in a two dimensional array.22. The firearm laser training system of claim 18, wherein the meansoperable to calculate a horizontal calibration value for each pixel inthe third image further comprises: a means operable to store thehorizontal calibration value for each pixel in a two dimensional array.23. The firearm laser training system of claim 18, wherein the meansoperable to calculate a vertical calibration value for each pixel in thefourth image further comprises: a means operable to store the verticalcalibration value for each pixel in a two dimensional array.